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(57) Abstract 



Multi-media networks will require that a data flow be given certain quality-of-service (QOS) for a network connection but 
pre-negotiation of this sort is foreign to the current data networking model. The real time traffic flow in the data network requires 
distinct limits on the tolerance to delay, and the variations in that delay. Interactive voice and video demand that the total delay does 
riot exceed the threshold beyond which human interaction is unacceptably impaired. The present invention allows the network to discover 
the nature of the service for each traffic flow, classifies it dynamically, and exercises traffic conditioning by means of such techniques as 
admission control and scheduling when delivering the traffic downstream to support the service appropriately. 
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DYNAMIC TRAFFIC CONDITIONING 

Field of the Invention 

The invention relates! generally to the conditioning of traffic at a 
5 node of a data network. In particular, it is directed to a technique by 
which the traffic ^ d^amically classified at a node and delivered 
downs trearri according to tfie Quality of Service (QOS) specified by the 
assigned class or by such a network admi^ as the network 



■- ; ^3ac^^ Y ■ - v : . Y 'vs' 

l^ultiriheclia rt| require that,a data flow be, given 

certain QOS for a nejhVork c6rir^ection:^^^T^ proposed resource 

.. ;} '; r ^^^P*?$$ (^$VP) sigriailirig in ATM networks) is 

' i5 f ™ ay °* ^ ^^cular QOS but pr6-negotiation of this sort is 

' J???^^ would r^cjuire 

^Y-^'.f^^ J.:,, ■ 

Y/\; ^ei majpf d^ for clu^ferent ' ■ 

. Q.°%, ^ ^ 4^ $$ty.9&- ^ need to introduce real time; flows 

• f ^?H§^ -^^^S^f 1 ^ i'^^ife^^ t^yp?^-i i 11 ?^^^' ^ tl: >!e^ human interaction is / . 

■ be kept to a rfeasonftle ■"" 

' '■ ' S P?, ^^?^teed npt to^ Meetmg the delay requirements 

these flows must be given . 
•i^or^ limit 
. ft e • ai ^? u P* ^ su ^ ¥fe P^o^ I? ^ficjby some admission control 
3? get some of the available 

■ \ ;;-i;ban^^d .;Y; : ;. ".X Y. X. YV'f' . 

ben ?^ % r ? 1 controlled latenc^. ^Network control traffic such as DNS 
■ tra^actions represent a small fraction of th? total but will provide a 
35 much improved performance if treated with priority. 

There is another class of traffic which does not have the tight 
requirements of voice or video but does involve human interaction 
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with computers and can lead to noticeable decreases in productivity (or 
increases in frustration) if subjected to long delays. This traffic type is 
generated by applications such as X-Windows> Telnet and, more often 
now, world wide web browsing. This traffic can be protected from long 
5 queuing delays caused by bulk transfer! sUch as FTP br NFS by 

M '^P^: p^^i0^ portibrt of ^e bandwidth as part of an 

■H^/^rbu^utisc^ : s\'-"' :! -" : - .: ■ • 

■' E y^V |he bu^ traffic can; suffer from tob much competition. . 
: O^en a file tra^fef mil be : 



• _ : apphcatipn or the ;#ser ; ' '^K^0§$^^i^^^^m-^^^^^ ' 

' ■' servers. Xisd> ^ : -. 

■ easily impact many flo^ys,^ a •, ; 
' : : icfertim number bf^ . . 

■ r . • . • : - It ; wpitf d ; ;£e^^ ' 
' ' " heed >:'. 

20 ' 



'f ^aditibna ' 

^ .'■ 'f, ^W&ti^ ' arid ,• ; • .! 

schbdulir^ when; . 



service app^ s^^u^^s^arates re^r tirne traffic: from 

bther traffic classes .' 

"'" of .traffic. - Jp. :coin^ ^ ' 

3 grantees perf^r^r^e; rT^ 

of overlay admirustratiye p^ groups .' 

: different treatment;*!^ need hot •■ 

:-emu|ate precisely ■ ; 

should provide- similar imprbvemerits in service quality as seen by the 
users and the network. s 
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It should of course be noted that in this specification the data 
network can also include any packet-based or cell-based networks, 
including ATM networks. 

5 Objects of Invention 

It is therefore ah object of the invention to provide a method of 
and an apparatus for dy^arnicajly cdnditib^ traffic at a node of a 
data network. V-. ' ];r f- y r-.;.' !r . ••' 

l.J.-}---' ft ^;|v^i^^? Pbj^ a method; of and 

10 apparatus ^ dyharnicaily classify^ 

^affic irjto ojh£ of according to a set of - 



.' ; • ft;^'^$^$^ method of arid 

apparah^ for cc^rrbU 

15 according to quality of service parameters specified by me dyna^ally 



20 



selected class. 



' f ;'!-'iC ', Bri 4?y ^^o^ Resides irt a packet data network for "' 

C- •^'^^? £ %^ classes 
25 atdo^m^ pre^|t crr^ The ihemod further 

■ ■ /'-S^ according to trie 

;-v/, ; ;; ; ^ality;ofsej^^ •'• 

method of dyriamicall^ coh^W pf • 

' 30 'j 6 *^"^ 1 ?-^^^ at a ri6de to detect a real time . 

; M^Y^^::&^w^1rca^ fij^^^rt 'a^;a^4ilaB.le band width of the data ' 
network. The me^q^ real 

I* 1 ?? a diff^ef ent bia^ delivery downstream according to a 

35 quality of service specified by the different class. 
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Brief Description of Drawings 

Figure 1 is a schematic diagram of the traffic conditioner 
according to an embodiment of the invention. 

Figures 2 and 3 show possible locations of traffic conditioners of 
5 the invention. 

Figure 4 illustrates that the traffic conditioning features of the 
invention can be installed at a router or a switch. 

Figure 5 is a pictorial view bf state ttansitibhs. 
Figure 6 show$ classificatioh mf ofmation bling propagated 
IP downstream.; : C-'':''^!: 'C ;: ^\TL '■■ ■' " V-" ' ' " 

Figure 7 shows that me REJEGT inf^prmation is being passed 
forward and reflected back to the source host. 

Figure 8 shp\y s a circ|ums tance in xyhich a s toj> message is sent 

■ , back upstream from a node. ' ' '.. •. 

DetaUed qescriptidn of the Pref erred ^^dM^td^ifihi}i^m^tm-- 

.. " Referring td .Figure 1, the traffic conditioner, according to one 
embodiment of the invention; includes a pIurMi^ of queues 10; at least 
. one for each class. It is located at a nbde of a data hetwork: Every 
20 packet of an iriput stream inspected and id^rttifi^d at 12 using; for 
; : example, IP addresses;; por^s "j&Sd ^^otpc^i-f-^A. ■is^ib»Iler'"l4 ! characterises 

■ the flow (losing rate/duratip^^etc.) and ^igriis it a tlass. A plurality of ' 
.■ classes are envisioned ac 

be described i^ter, Tnecontroiier ^ a- database 16 ind. uses ' 

25 . Output scneduHng to classes; arid 

implements an admission control poli^ delivering an . 

output stream toward dowri^tidar^ r^bdes or to peripherals; It also; 
discards, packeite fro 
;•, classificatfon ta i • :: ; ; 

The traffic conditioning p at 
various places in the ^iata Network. For examplef, as gafew^s are often ' • 
a bottleneck and; bulk flows can decrease response t^es for interactive 
users, a traffic cdnditioner can; b^ 2 
which will alleviate this problem.: In Figure 3/ traffic conditioners are 
35 located at a plurality of IP switches which form a data network 40. 

Packet switches will not be able to provide good performance for hew 
services/such as real time video, unless traffic conditioning is 
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implemented. A conditioner at an output port of a switch 42 monitors 
and controls all the traffic which comes into the switch and goes out of 
this output port to the downstream node 44. An integrated traffic 
conditioning feature can be installed at a router or a server as shown in 
5 Figure 4. 

In one instance of IP networking, the identifying individual 

, I^otocpl pprt nu^be^ f^ bp^ source and destination. This approach 
ha^ been ^ applications such ^ jqp Routing 

%0 and, rhor^ recerltly, in IP ^witdiirig. 

Any scheme w^ and classifies flows for differing 

. ., ■ *? ea * m ^ to suit the particular ,." 

,; _ '. typical TGP^EP based 
15 network are cbnsiciered and ides^ibed in detail below. 

(Class 1) - Interarriye I Jspre fTY^P} 

^^•P^'&mt^^ applications 
as X-wind^w^, Telnet a^ Hgh^ight we> browsing. Telnet and X- 
■ 20 ... W ^f?°™ s can be very longJSes&dn^ but have predominantly short " 
packets. M &i£ emb<^ classification defines a short packet 

25 image files.: This dass is the d but when the 

fl6w exceeds a number (e.g;, ?00) of consecutive long packets, it is 
considered to be bulk transfer Md no lo^ this class. 

(Class 2) - Bulk Transfer With Guaranteed Band wid fTT.Pl 

If TCP traffic is not in^ 
§ome ?f tjtefcu}^ a scheduling class . 

■ $ P^^ted>PPrHdiy^ bandwidth and a limited number 
of members such that some mmimum bandwidth is guaranteed for 
these flows. 
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(Class 3) - Bulk Transfer, Best Effort (TCP) 

Any bulk TCP flows which are not admissible to the guaranteed 
bandwidth class are treated in this class on a best effort basis. Classes (2) 
and (3) are intended to handle large FTP or large web images. 

5 

(Class 4) - Low T.atency CI JDP) 

This class m 
general, these flows consist of I6W bandwidth voice, NFS requests, 
short NFS replies, arid network contrbi ^ckets such as "DNS , 
10 transactions. Most real time voice Will be contmuous flow UDP (User 
Datagram Protocol) at less than 20 packets per second. This is the 
default class for UDP an^ 

moved out fast; There wbuld be an admission control policy such that 
if the allocated bandwidth fortius class is used up, hew flows would be 
15 moved to the best effort class. 

• . (Class 5) - Real f ime (lIPPV L •U-.'.y-,-.j: :'v: ;■"■>.■.: v.: 

Any flow with real time characteristics Which will be defined 
below and a baridw 
20 captured in this class. High baindwidth voice and streaming video are 
the expected members. For example; video Will be a continuous UDP 
flow at up to 4Mb /s = (lOOOpps): Flows exhibiting real time 
characteristics but for which there is 'rib >\resotfr ce imder the Admission 
control policy, would be rejected and subsequent packets are discarded. 
25 ... '•• ; , ,. .' •/ :v-\ .:';^.i'>-i : .k,. ■' 

(Class 6) ^ Bulk Best Effort fUDPV ^ : > — 

All other UDP floWs will be grouped into this class. Expected 
members are NFS file copy and backup sessions characterised by many 
long packets at a substantial raited 

In summary, according to this embodiment, discrimination 
between traffic types is based on simple analysis of packet arrival rates 
and packet length plus a particular test for real time UDP traffic which 
will be described below. Although it is possible to use port numbers in 
some instances to determine service type, this method is not consistent 
enough for general use. The characteristics of a flow are monitored 
continuously and the flow can be re-classified during its lifetime. The 
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ability to re-classify also covers the cases where consecutive flows 
assume the same identity but have different characteristics. 

Figure 5 is a pictorial view of state transitions involving these 
classes according to the embodiment. In the figure, the following 
5 abbreviations are used: 

p = packet 

sp . : = v :Shprt'pa^ • : ' ; 

lp = long packet 

con lp = consecutive long packets 
10 bw = b^d^dth available 

chosen = chosen by policy 

PPs = packets per second arrival 
J ; ,. . ^eioUp\^^ • - 

(a) For all classes, return to' initial if there are no packets for 30 
15 seconds. ' : '-/"V ■ '■ . v:' :: 

M. *° r /^$Q^^ is defined then fofce 

the flow to that. clkss for 100 packets or 30 seconds. € 
Referring tp Figure 5, thie state, machine contains the following 

states: 
20 (A) Initial 

(D) .TCP Bulk Giia^anteed 

(E) UDP I^w Latency 
25 (F) LTDP Real Time ; ., - 

(G) UDP Bulk Best Effort '. 

(H) Reject 

; ^ the, followmg detail^ description, ">" indicates "true" for all 
the states. • 

30 ... .• . ,. ' .; _ , - , : 

Power up 

> [Idle for 30 seconds J-> Initial 

If class information is received frpm upstream then force that state for 
.35 100 packets. After 100 packets or 30 seconds with no information, 
revert to local judgement. 
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> Initial (at 50 in Figure 5) 

> [Protocol=TCP] -> TCP Interactive 

> [Protocol=UDP] -> UDP Low Latency 

5 > TCP Interactive (at 52) 

> [200 consecutive long packets ':&^- ! 6f 'size > 512 bjrtes] 

> [Some TCP Bulk^ min bSnfdwidth available] 

~> TCP Bullc Cvuaranfeecl 

> [O'tn'e'rwiseJ ' ' ' :; ' : ' ' : ' " ''' 1 ' 
10 --> TCP Bulk Best Effort 

> TCP Bulk Best Effort (at 5iX' ; . ' i , ; 

> [Departure of a ^)rTd^"£i^3axi^ TXi^^Bxiiic" W£^f&i^i : - ( aA^; BPatoJoecl" v- 

a Policy] ; : , y^Wt'.y- ':" "-.y ; ' 

15 ~> TCP Bulk Guaranteed 

• > [Two consecutive short pac^ than 512 

' : .bytes]'v 'v ■■■'•^ ■ '>■'''■' ■ S-'' 

--> TCP Interactive 

20 > TCP Bulk Guaranteed (at 56) - 

> [Two consecutive short packets of size less! than 512 bytes] 

— > TCP. Interactive- • ^•* / : ; •''•.'^ ' 



25 



> UDP Low Latency (at 58) 

> [After any 1 second in the state, and > 25 packets have arrived] 
—> UDP Bulk BestlEffort 

or '" .'■ \ : :-' :X Jj^ . ' =' ;■ . ; . 

> [Two or more cpii^ectitive long packets] and [>5 packets in 1 
second] V ■■; '-r^i. -y.^ , ■ ; , 

30 -> UPP Biilk Best Effort 

>[Band width is known] a?id [no more bandwidth available] 
--> UDP Bulk Best Effort 

> UDP Bulk Best Effort (at 60) 

35 > [Matches real time template] and [bandwidth available in UDP 

video] 

--> UDP Real Time 
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> [Otherwise] — > Reject 

> [More than two consecutive short packets] and [bandwidth 
available] 

~> UDP Low Latency 

5" - T" ' ' ' 

> UDP Real Time (at 62) 

> [300th packe^ 200 ms] 

//^.ro^;;;LbwLal^c^.'. : r 
- > [Wle 1 secbnd}^,^ ; r'- 

. : .... • , : ; ^; ; R^f|:t:^^g5 



15 



exce 

information; 



20 



Real Time Flow rWprKnn v-,.'.'; .-.V*!- 

25 ' ^^P*?^.^ Ickr^Wledged burst (tne "'' 

" '■^^^SE^^ times fortKe 

' , .: a^and^the average •' ' 

' 3 ° ?ate, using a 

., 

. . *° r afriyal tiW^ If the first copht is more than; 

10% of ^e second, count then^ flow is ^ 
35 second method may be pre^rre;d smce it clbes hot require waiting for 
congestion to id^tify the stream. Quick admission or rejection of the 
flow is important. 
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Scheduling Flows 

There are various techniques available for scheduling multiple 
queues at output ports of a switching node. In any case, typically, the 
real time and low latency classes described above would be given 
5 absolute priority (hence the need for admission control), arid the other 
classes would be schedule to th^m the 

• proportion of 

Referring tb Rgure; : ^ instead ^ each 

' ho ^ P#forming -th^ ;', 

"■' 10 ' "^^^f^^^W^W^^ : P^ its findings to •' .' 

the dbwhstrea^ n^ 

classifies me tr^ 



■;e^ire^anclpr(^ 

'2^./ ^value'C^ i-'- y ^;^(\fi :i \\ ■ .; ' .'7'';' : ? ■ •' 

• '. the -^tiria^ thus utiprpyi^ the 

■ V ; efficiency; ^'the;|iet^M^ : . 

7p '^.d^termines ^at^e\p|icket-m '{ 
■ d^aixis ail but eye^^trvp^ destination 
host, which m 

another scenario as shown 8/instead of. the ^ 

35 which rejects the traffic stream for some reason can send a message 
back upstream indicating that the traffic has been rejected and should 
therefore be stopped. 
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Complexify 

In any network where different qualities of service are being 
supported, there has to be some process to classify the flow attributes 
and then a way of yerifying that the flow is adhering, to the attributes. 
5 This verification, or policing, can be done at the edges of a subnet or at 
e X^y nocte. The : ^0i^:^^d^^-^S^^u^\^ mechanisms that 
•• ?n Pj^M? ® ^j^ver ^ How. attributes! but 

... V ®^S!?^-?^^ ; . 

v.: , Ttbdes 

The adihii^^ policies on bandwidth 

allocation t^t^ yario^ 

' '.. - ■ ■ :S 6 ~^*f : j^^^^^l;' ^^^^ ^Connections ' 

• sig^ailin^, i^ M*lic>;p<>sS^ as ' 

, a ^uk oflugher lewl^ a 

:' ..v.' ; .-. ' Ikb^SS^ can- ■ ' 

;' I ■ :'. * be ^isable^^ marketfMo^. .. 

■ AecoM^g to&^ flows are , "f 

. . Wlity.;6f s%^ appUcation ." 

": s *S?#*Hs s^oit i&- no£ is no ihore ' • 

cb ^* e * than that ^equir^d for ^ when signalling is used and 
£ e ^ ^^fef^M ^^^^W^ K^c^^cl network performance : 
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WHAT IS CLAIMED IS: 

1. In a packet data network for multimedia traffic having one or 
more nodes, a method of dynamically conditioning traffic Comprising 
5 steps of: - ■■■ 

continuously monitoring the traffic at a node as to its traffic 
' characteristics; and; ; : ■ '■''■:"■■.■'] ■ '-, ■"<"■,.■■■/ 

classifying tte traffic into one of a plurality of classes according to 
preset criteria of traffic: <^aracteristics; skid pfaraHfy-df claises^cifyihg 
10 respective levels of the quality of service wim \vhich to condition the 
traffic. : /^s.v' i--:. ' ^'i ■' 

2. The method according t6 claim 1, ^^inAe preset criteria are 
protocol types of the traffic> the protocol ryp^ 

3 * The method accpriiihgito cfaim 2 whereiri the step of classifying 

-^f 1 ^ * s * e P $ classifying V^to two defa^^ interactive 
• and IJT)B low latency: ' f .'Vv *' ['■*."'■ "' 

20 4 r The method according to claim 3 cornprising a further step of : 
changing the TCP interactive class to one of theother of TCP 
bulk guaranteed artdfTGPt^b^^ ■V\- ,y :'' ■■' 

, ; r , 5. The method >' d'o^iS^ijr^' to- iliauim .3 - Vfi^^i^.lBt!^"" ofi 

2 - ; changing the UDP iow : latency class to one or the other of UDP 
real time and UDP bulk best effort. 

6. The method according to claim 4 comprising a further step of: 
delivering the traffic downstream according to the quality of 

30 service specified by said one o^ 

7. The method accordirig to claim 5 comprising a further step of: 
delivering the traffic downstream according to the quality of 

service specified by said one of a plurality of classes. 



35 



8. the method according to claim 6 wherein one of the preset 
criteria is the length of packets in the traffic. 
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9 - The method according to claim 7 wherein one of the preset 
criteria is the length of packets in the traffic. 



5 



10. The method, according to claim 6 wherein one of the preset 
criteria is the ntf^ \ 

The me^c| ! ^^f d^ the preset 

'criteria-, is '''the hiirnker ^f consecutive pkfekjts in the traffic, ' \ 



■}°:'\^(: : quahty pf service ' 

of delivery of th# traffic dpWh^ arid the bandwidth assigned to the 

' • . traffic^ ; ' '.- 'i^^^f^,.^^ , . . ; • \, ; ■ : \. v - v>i - 

15 , 13 / ' ;.^^:V^^^^^^-^^la^ icbrhprismg: a furtr^r step^ of: ? 

. ; M< The m^ to claiirt 13 wherein the step of mforrning 

20- v c6ihprises a step^ofr"' "' ' ' : \-"' " • '• : -' ■■ 

: v|>rp^ in a pkcket which is 

i^-' jh;3' pa^ traffic havingone or ' 

25 more nodes, a v m|tht)df of jtiyna^^ comprising 
steps of: ; 'V . 

continuously m0m^ traffic a| to its traffic characteristics 

. a s< ?"^5- v 1 ^! v%^. ttaf Ac is generated; and 

. tfe ^ffic into one of a plurality of 

0 classes accprcling to preset criteria of traffic diaracteristics, said plurality 
of classes specifying ;res^ of the quality of service with 

which to ccaiditioh fc 

16. The method according to claim 15 comprising a further step of: 
5 _ delivering the traffic downstream according to the quality of 

service specified by said one of a plurality of classes. 
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17. The method according to claim 16 comprising a further step of: 

m ^ ormin S Qne ? r ^ ore downstream noiies the results of 
monitoring and classifying^ 

5 18 - ' m^ the step of mformin£ 

comprises a step of: 



: : ,? ..P^gawig. uif^ttpn in -a packet whiph is. 
'i ? /"ir^er^oii ' 

10 ' 



cp^tini^^ node to defe^ a real 

time traffic flow; . . "i v> ; ^- ^ '/' '.; i ^ :'' "'V. 

15 : v ;;; ■ -^/^'jf????^^ for;! - . 

• •■ ■ deii\^r^<i6whls1^ of the data ■• ' ■ ■ 

network; arid ' >',Y. ^,Y^--. : -' v ? .•' 

■ • .r^ciassifyM^ ...";'•"*•"' 



20. class. 



2 P- Themeth^ 

continuously monitoring thei^ffic further doniprises a step of: 
... ■ ^ at the 

,?;25 ; node mto ^hicK the traff^ ' 

. ; 21. The method accor<ding to daim 19 WHerem th 

continuously mbmtbring^^ a^tep of: 

monitoring a Wstory of interarrival times for packets of the 
. 30 traffic, ~ ; ^Z;-, ^Sry'i&fiyi-H: v: -■■ .'\ ■ ; ' - - 

22. The method according to claim 21 comprising ^ further steps of: 

recordmg a first count for the arrival times below a preset 
threshold; 

35 recording a second count for the arrival times above the 

threshold; and 
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determining the traffic as being a real time traffic flow unless the 
first count is more than the second count by a predetermined amount. 

23 r The method according to claim 19 wherein the quality of service 
5 is expressed by a plu^ a priority level 

?f delivery bf^ ^^ii^ dow^h^^ arid the bandwidth assigned to the 



24. t^ me^od according ^daim l^bmprisin^ a further step of: 
10 informing 6ne or more dciv^rf earn nodes the results of 

monitoring and ciassifyihg. 

25 ^ e *^9 d ^ccbrding to clami 24 wherein the step of informing 

/-.>// ; c^^ ■ 

^ P r °F^g a ^g^£ packet which is . '•" V 

inserted irifo the fralffic; f ' •• : V': :; :''.>;': 
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